Skip to content

⚡ perf(import): defer platformdirs imports - #722

Open
gaborbernat wants to merge 2 commits into
mainfrom
perf/lazy-imports
Open

gaborbernat wants to merge 2 commits into
mainfrom
perf/lazy-imports

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Common platformdirs calls previously loaded helpers they did not use. In a paired 60-run Python 3.15 hyperfine benchmark, package import measured 25.1 → 17.3 ms.

Module declarations defer eligible imports until needed, and flake8-lazy checks future imports. Android cache handling and Unix runtime helpers stay eager because callers can observe and patch those imports. Earlier Python versions keep eager imports.

Benchmarks

hyperfine 2.0.0 --warmup 5 --runs 60 compared the PR with its merge base. Each pair used the same interpreter and dependencies in a tox environment; PYTHONPATH selected the source tree. Python 3.14 does not enable lazy imports. These wall times include process startup. The Python 3.14 CLI help comparison has high variation, so its apparent slowdown is inconclusive.

Python 3.15

import:

Benchmark 1: env PYTHONPATH=/tmp/lazy-bench/platformdirs/before/src /tmp/oss-lazy-platformdirs/.tox/3.15/bin/python -c 'import platformdirs' (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     25.1 ms  ±     3.9 ms      21.0 ms  …    37.7 ms
  Memory        13.8 MiB ±     0.2 MiB     13.3 MiB …    14.2 MiB
 
Benchmark 2: env PYTHONPATH=/tmp/oss-lazy-platformdirs/src /tmp/oss-lazy-platformdirs/.tox/3.15/bin/python -c 'import platformdirs' (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     17.3 ms  ±     2.9 ms      14.0 ms  …    25.8 ms           -30.9%  (1.4x faster)
  Memory        11.5 MiB ±     0.1 MiB     11.1 MiB …    11.7 MiB          -16.3%

help:

Benchmark 1: env PYTHONPATH=/tmp/lazy-bench/platformdirs/before/src /tmp/oss-lazy-platformdirs/.tox/3.15/bin/python -m platformdirs --help (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     26.9 ms  ±     3.8 ms      22.7 ms  …    41.3 ms
  Memory        14.3 MiB ±     0.2 MiB     13.7 MiB …    14.5 MiB
 
Benchmark 2: env PYTHONPATH=/tmp/oss-lazy-platformdirs/src /tmp/oss-lazy-platformdirs/.tox/3.15/bin/python -m platformdirs --help (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     22.5 ms  ±     3.9 ms      18.2 ms  …    33.5 ms           -16.5%
  Memory        12.4 MiB ±     0.1 MiB     11.8 MiB …    12.6 MiB          -13.3%

Python 3.14

import:

Benchmark 1: env PYTHONPATH=/tmp/lazy-bench/platformdirs/before/src /tmp/oss-lazy-platformdirs/.tox/3.14/bin/python -c 'import platformdirs' (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     25.3 ms  ±     4.9 ms      19.4 ms  …    37.2 ms
  Memory        14.8 MiB ±     0.2 MiB     14.1 MiB …    15.0 MiB
 
Benchmark 2: env PYTHONPATH=/tmp/oss-lazy-platformdirs/src /tmp/oss-lazy-platformdirs/.tox/3.14/bin/python -c 'import platformdirs' (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     23.3 ms  ±     3.2 ms      18.6 ms  …    36.8 ms            -7.9%
  Memory        14.8 MiB ±     0.2 MiB     14.1 MiB …    15.0 MiB            0.0%

help:

Benchmark 1: env PYTHONPATH=/tmp/lazy-bench/platformdirs/before/src /tmp/oss-lazy-platformdirs/.tox/3.14/bin/python -m platformdirs --help (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     25.2 ms  ±     4.0 ms      20.3 ms  …    39.9 ms
  Memory        15.2 MiB ±     0.2 MiB     14.7 MiB …    15.5 MiB
 
Benchmark 2: env PYTHONPATH=/tmp/oss-lazy-platformdirs/src /tmp/oss-lazy-platformdirs/.tox/3.14/bin/python -m platformdirs --help (60 runs)
                mean     ±       σ          min     …     max
  Wall Time     31.4 ms  ±     8.9 ms      21.7 ms  …    55.8 ms           +24.7%
  Memory        15.2 MiB ±     0.2 MiB     14.6 MiB …    15.4 MiB           -0.1%

Import-time profile

Lifeguard scanned the source tree with --python-version 3.15. I also imported 13 modules separately with python -X importtime.

These are one-run cumulative import timings for the modules with the largest reductions:

Module Before After
platformdirs.android 21.3 ms 8.5 ms
platformdirs.windows 22.2 ms 11.0 ms
platformdirs.macos 17.4 ms 8.5 ms

The Python 3.15 package import diagrams use python -X importtime -c "import platformdirs" 2> import.log and tuna.

Before

Before import tree

After

After import tree

Declare imports safe to defer on Python 3.15 and check new imports
with flake8-lazy. Keep cache and patch targets eager.
@gaborbernat
gaborbernat requested a review from ofek as a code owner November 18, 2026 07:32
@gaborbernat gaborbernat added the enhancement New feature or request label Nov 18, 2026
@gaborbernat
gaborbernat marked this pull request as draft November 18, 2026 07:46
@gaborbernat
gaborbernat marked this pull request as ready for review November 18, 2026 08:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant